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Introduction 

Autism  spectrum  disorders  (ASDs)  are  disorders  of  the  brain  that  affect  as  many  as  1  in 
88  children.  Without  intensive  and  appropriate  treatment,  the  long-term  outcomes  for 
children  with  ASDs  remain  bleak  and  are  associated  with  a  high  divorce  rate  among 
parents  and  increased  risk  for  mental  health  disorders  among  family  members.  The 
efficacy  of  and  empirical  support  for  interventions  for  ASDs  based  on  applied  behavior 
analysis  (ABA)  is  well  documented.  Unfortunately,  these  services  are  often  not  available 
to  military-dependent  children  because  there  are  not  enough  appropriately  trained 
individuals  to  design  and  provide  ABA  services.  This  project  will  demonstrate  how  web- 
based  technologies  can  increase  the  availability  of  effective  treatment  for  military- 
dependent  children  with  ASDs.  By  evaluating  a  technology-enhanced  treatment 
delivery  model,  military  families  will  be  able  to  receive  empirically  supported  treatment 
services  in  a  timely  manner  anywhere  in  the  world.  Also,  training  therapists  to 
implement  ABA  programs  using  a  web-based  model  will  greatly  increase  the  number  of 
well-trained  therapists  in  areas  around  many  military  bases. 
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Body 

Task  2.  Complete  parent  and  tutor  training  and  EIBI  aims  for  5  cohorts,  which  includes 

50  participant  families  and  their  ABA  tutors  (timeframe,  Months  12-44). 

From  Statement  of  Work:  Tasks  include  three  distinct  sub-tasks  for  each  cohort.  Our  training 
protocol  based  on  E-Learning  using  the  latest  web-based  and  televideo-based  instruction  will 
provide  an  efficient  and  effective  mechanism  for  training  military  parents  of  children  with 
autism  anywhere  in  the  world  by  experts  in  one  location  (University  of  Nebraska  Medical 
Center  in  Omaha).  These  trained  parents  will  then  be  able  to  implement  effective  behavior 
management  and  teaching  strategies  with  high  procedural  integrity  (90%  accuracy).  Second, 
we  will  use  the  protocol  we  developed  to  train  adults  to  become  ABA  tutors  and  to  implement 
early  intervention  procedures  with  high  integrity  (90%  accuracy)  in  areas  of  the  world  where 
such  services  are  unavailable.  The  training  protocol  (using  web-based  E-Learning 
instructional  methods)  will  be  the  same  for  the  parents  and  the  ABA  tutors,  but  they  will  have 
more  extensive  curricula  (with  the  curriculum  for  the  ABA  tutors  being  more  comprehensive). 
Third,  we  will  evaluate  changes  in  cognitive,  language,  social,  play,  and  adaptive  skills  and 
decreases  in  problem  behaviors  among  children  with  autism  in  military  families  who  receive 
technology-enhanced  early  intervention  services  that  are  supervised  by  our  experts.  The 
children  in  the  technology-enhanced  early  intervention  treatment  group  will  show  significantly 
greater  improvements  than  children  randomly  assigned  to  a  waitlist-control  group. 

In  each  cohort,  there  will  be  10  children  with  autism  recruited  with  at  least  one  parent 
(cohort  n  =>  10)  and  one  ABA  tutor  (cohort  n  =>  10)  per  child.  In  the  first  cohort,  5  of  the 
children  and  their  corresponding  parents  (n  =>  5)  and  ABA  tutors  (n  =>  5)  will  be  randomly 
assigned  to  the  technology-enhanced  early  intervention  treatment  group  and  the  other  5 
children  and  their  corresponding  parents  (n  =>  5)  and  ABA  tutors  (n  =>  5)  will  be  assigned  to 
the  waitlist-control  group. 

We  have  reported  progress  for  the  preliminary  evaluations,  Experiment  2,  and 
Experiments  1  and  3  separately  to  improve  the  organization  of  the  annual  report. 

1.  Preliminary  evaluations  on  the  efficacy  of  the  parent-training  curriculum  designed  to 
teach  parents  in  the  technology-enhanced  group  (Experiment  1). 

a.  We  evaluated  our  technology-enhanced  parent-training  procedures  on  teaching  a 
parent  how  to  increase  her  child’s  communication  skills.  Below  is  a  graphical 
depiction  of  one  parent’s  performance  in  learning  how  to  teach  their  child  to 
request  a  highly  preferred  item  (i.e.,  access  to  an  IPAD).  The  top  panel  depicts 
the  parent’s  performance  across  all  1 1  skills  (closed-circles  plot)  and  the  bottom 
panel  depicts  the  parent’s  performance  for  each  of  the  individual  skills  (black, 
gray,  and  white  boxes  denote  accuracy  levels  of  90%,  80%,  and  below  79%, 
respectively). 

Prior  to  experiencing  our  training  curriculum,  the  parent  only  exhibited  about  half 
of  the  target  skills  and  did  not  correctly  implement  several  critical  teaching 
components  following  an  incorrect  response,  which  included  providing 
instructions,  modeling  the  correct  response,  practicing  the  skill  with  the  child,  and 
ignoring  the  child’s  problem  behavior  (these  component  skills  with  errors  are 
highlighted  by  the  red  boxes). 
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After  viewing  our  60-min  multimedia  module  on  Preventing  Problem  Behavior  and 
participating  in  scripted  role-play  sessions  with  an  expert,  a  robust  improvement  in 
the  parent’s  performance  was  observed  during  the  first  post-test  assessment 
(session  2)  and  this  improvement  was  maintained  across  a  second  post-test 
assessment  (session  3). 


Figure  1. 
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b.  We  also  evaluated  the  parent-training  procedures  on  teaching  two  parents  to 
implement  discrete  trial  training  with  their  child.  The  data  in  Figure  2  below  shows 
the  relation  between  parents’  accuracy  of  implementing  the  teaching  strategies  as 
designed  (i.e.,  procedural  integrity)  and  their  child’s  learning  outcomes.  In 
baseline,  parents  were  asked  to  teach  their  child  an  expressive  identification  skill 
(e.g.,  stating  the  color  of  different  items)  using  discrete  trial  training,  based  on 
written  instructions  of  the  procedures.  On  the  y-axis  is  the  percentage  of  teaching 
opportunities  in  which  the  parents’  correctly  implemented  the  teaching  procedures 
and  their  child’s  correct  responding.  Both  parents  implemented  only 
approximately  one  third  (top  panel;  closed  circles)  to  half  (bottom  panel;  closed 
circles)  of  the  teaching  procedures  correctly  and,  as  a  result,  their  child  did  not 
acquire  the  target  skill  (white  triangles).  Following  our  web-based  training,  in 
which  the  parent  received  real-time  feedback  on  implementing  discrete  trial 
training,  the  percentage  of  opportunities  with  correct  implementation  of  the 
procedures  increased  to  above  60%  for  the  parent  in  the  top  panel  and  80%  or 
above  for  the  parent  in  the  bottom  panel.  This  led  to  an  increase  in  their  child’s 
performance  to  nearly  100%  accuracy  and  60%  accuracy  in  the  top  and  bottom 
panels,  respectively.  After  some  additional  training,  the  parent’s  accuracy  in  the 
top  panel  increased  to  above  80%.  We  observed  maintenance  of  both  parent  and 
child  performance  during  the  post-training  assessment.  Experimental  control  over 
the  effects  of  the  training  curriculum  was  shown  using  a  multiple-baseline  design 
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across  parent-child  dyads.  These  outcomes  are  examples  of  the  efficacy  and 
maintenance  results  we  expect  with  parents  in  the  randomized  clinical  trial  as  we 
continue  to  increase  the  number  of  participants  enrolled. 

Figure  2. 


Training 

Pre-Training  Probes  Post-Training 


2.  Preliminary  evaluation  on  the  efficacy  of  the  tutor-training  curriculum  designed  to  train 
ABA  tutors  in  the  technology-enhanced  group  (Experiment  2). 

We  evaluated  our  tutor-training  curriculum  on  teaching  adult  tutors  how  to  conduct  a 
standard  preference  assessment,  which  is  commonly  used  to  objectively  identify 
children’s  preferences  who  exhibit  limited  communication  abilities.  Below  in  Figure  3 
is  a  graphical  depiction  of  three  tutors’  performance  in  learning  how  to  accurately 
implement  the  13  skills  involved  in  conducting  the  preference  assessment.  Each 
panel  represents  a  different  tutor’s  performance,  with  respect  to  the  percentage  of 
component  skills  implemented  with  accuracy  above  90%.  The  open  circles  denote 
sessions  conducted  with  a  child  with  ASD;  the  closed  circles  denote  sessions 
conducted  with  a  “simulated”  child  (i.e. ,  an  adult  who  exhibits  child-like  behaviors). 

Prior  to  experiencing  our  training  curriculum,  the  tutors  were  given  written  instructions 
on  how  to  implement  the  preference  assessment.  All  three  participants  performed  at 
unsatisfactory  levels  with  written  instructions  alone  (i.e.,  below  50%  accuracy).  Next, 
we  implemented  our  web-based  training  procedures,  which  involved  the  tutors 
viewing  a  60-min  multimedia  module  on  Preference  Assessments  and  participated  in 
a  scripted  role-play  session  with  one  of  our  experts.  After  viewing  the  module  and 
completing  one  role-play  session,  a  notable  increase  in  the  percentage  of  component 
skills  implemented  accurately  was  observed  for  all  three  tutors.  Experimental  control 
over  the  effects  of  the  training  curriculum  was  shown  using  a  multiple-baseline- 
across-subjects  design. 

After  observing  consistent  performance,  we  conducted  an  additional  probe  with  a 
“simulated”  child  and  an  actual  child,  and  then  provided  one  more  session  with 
additional  training  (Partial  Training)  on  the  few  component  skills  that  were  more 
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difficult  for  the  tutors  to  acquire.  We  then  conducted  two  follow-up  observations  after 
a  month  and  observed  maintenance  of  the  tutors’  accuracy  in  implementing  the 
preference  assessment.  These  outcomes  are  examples  of  the  efficacy  and 
maintenance  results  we  expect  with  other  tutors  during  the  randomized  clinical  trial. 
Figure  3.  r  n  . 

Full  Partial  Maintenance 

...  .  Training  Training  Probes 

written  ,  °  ,  \  ,  s 

.  .  Probes  I,  (1  month) 

Instruction  V  V 


3.  Recruit,  pretest,  and  train  5  tutors  in  the  technology-enhanced  test  group  and  5  tutors 
in  the  waitlist-control  group  across  two  cohorts.  Schematic  1  details  the  progress  for 
the  technology-enhanced  tutor-training  evaluation  (Experiment  2). 

Schematic  1. 

Tutor  Training 

-  Randomized  Clinical  Trial 

-  Completed 

- Test  =  2 

- Control  =  2 

- 6  Enrolled 

- Test  =  3 

- Control  =  3 
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Single-Subject  Design 
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a.  Randomized  Clinical  Trial.  We  have  initial  outcome  data  for  the  between-subjects 
comparison  with  two  tutors  randomized  to  the  technology-enhanced  group  and 
two  to  the  waitlist-control  group.  Currently,  three  randomly  assigned  tutors  are 
progressing  through  the  technology-enhanced  training  while  three  other  tutors  are 
in  the  waitlist-control  group.  We  have  enrolled  7  additional  tutors  who  are  at 
varying  stages  of  completing  the  pre-tests  prior  to  their  randomization.  In  Figures 
4  and  5,  we  present  the  initial  outcome  data  and  the  results  from  preliminary 
statistical  tests. 

Dates  are  depicted  on  the  x-axis,  and  the  tutors  assigned  to  the  test  and  waitlist- 
control  groups  are  depicted  on  the  y-axis.  The  black  and  gray  horizontal  bars 
show  the  time  to  recruit  each  participant,  obtain  consent,  and  complete  the  pre¬ 
test  assessments.  After  the  pre-tests  were  completed,  the  participants  were 
randomly  assigned  to  either  the  test  or  waitlist-control  group.  The  red  circles 
denote  the  tutors’  performance  on  one  of  our  primary  dependent  measures, 
Behavioral  Implementation  Skills  for  Work  Activities  (BISWA),  and  the  green 
squares  denote  the  tutors’  performance  on  our  other  primary  dependent  measure, 
Behavioral  Implementation  Skills  for  Play  Activities  (BISPA). 

During  the  pretests  for  both  measures,  moderate  to  low  levels  of  the  component 
skills  was  observed  for  all  tutors  in  Figures  4  and  5.  For  the  tutors  assigned  to  the 
technology-enhanced  group  (P006  &  P009),  the  dates  for  completing  the  training 
modules  and  scripted  role-plays  sessions  are  denoted  by  the  gray  and  black 
circles,  respectively;  during  this  time,  the  tutors  (P003;  P008)  in  the  waitlist-control 
group  did  not  experience  the  training  curriculum.  Following  the  tutors’  completion 
of  the  40-hour  curriculum,  we  conducted  a  second  BISWA  and  BISPA 
assessment  (posttest).  The  participants  in  the  test  group  improved  from  an 
average  score  of  57%  correct  on  the  BISWA  on  the  pretest  to  an  average  score  of 
100%  on  the  BISWA  posttest,  whereas  the  participants  in  the  control  group 
showed  only  a  1  %  improvement  from  the  pretest  to  the  posttest.  On  the  BISPA, 
participants  in  the  test  group  improved  from  an  average  score  of  0%  during  the 
pretest  to  an  average  score  of  100%  during  the  posttest,  whereas  participants  in 
the  control  group  showed  no  improvement  from  pretest  to  posttest.  The 
differences  between  the  test  and  control  groups  approached  statistical 
significance  for  the  BISWA  (p  =  .066)  and  the  differences  for  the  BISPA  were 
highly  significant  (p  <  .0001 ).  We  anticipate  that  differences  between  the 
experimental  and  control  groups  will  all  be  highly  significant  as  more  participants 
are  added  and  our  power  to  detect  differences  increases. 
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b.  Single-Subject  Design.  For  tutors  that  withdrew  from  the  randomized  clinical  trial, 
we  are  evaluating  the  effects  of  the  technology-enhanced  training  within  a  single¬ 
subject  experimental  design.  This  evaluation  is  currently  being  conducted  with 
three  tutors. 
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c.  Social  Validity  Outcomes.  Following  completion  of  the  technology-enhanced  tutor 
training,  we  ask  tutors  to  provide  closed-  and  open-ended  responses  (ratings  and 
comments)  regarding  their  acceptability  of  the  web-based  technology,  content 
covered  during  training,  interactions  with  the  consultant  and  scheduling,  and 
overall  impression  of  the  training  program.  In  Table  1 ,  we  report  results  for  the 
one  tutor  who  has  completed  the  questionnaire;  the  tutor  used  a  7-point  Likert 
scale  with  the  following  ratings:  7  =  strongly  agree,  4  =  no  opinion,  1  =  strongly 
disagree. 


Table  1 

Social  Validity  Assessment  and  Results  for  Technology-Enhanced  Tutor  Training 


Questions 

Ratings 

Web-Based  Technology 

1 .  Prior  to  interacting  with  my  consultant  and  conducting  virtual  meetings,  you 
were  asked  to  setup  some  technology.  1  was  satisfied  with  the  process  of 
accessing  the  GotoMeeting  or  Adobe  Connect  meeting  room  and  starting  my 
audio  and  webcam? 

7 

2.  We  conducted  virtual  meetings,  using  either  Gotomeeting  or  Adobe  Connect, 
to  review  the  goals  for  the  upcoming  week  and  provide  a  status  update.  1  was 
satisfied  with  the  reliability  (e.g.,  consistent  internet  connection)  and  the  audio 
and  video  quality  of  these  virtual  meetings. 

7 

3.  Virtual  meetings  were  also  conducted  for  teaching  during  role-plays.  I  was 

satisfied  with  the  reliability  (e.g.,  consistent  internet  connection)  and  the  audio  7 

and  video  quality  during  the  role-plays. 


4.  We  provided  the  modules  and  quizzes  through  Blackboard’s  software 

platform.  I  was  satisfied  with  Blackboard  as  the  method  to  deliver  the  5 

modules  and  quizzes. 


Content  Covered  during  Training 

5.  Each  module  covered  the  core  content  of  a  given  topic.  I  was  satisfied  with 
the  amount  and  type  of  information  covered  in  the  modules. 


6.  We  conducted  several  role-plays  throughout  the  training  and  each  role-play 
covered  a  different  topic.  I  was  satisfied  with  the  role  plays  as  a  component  of 
training  (i.e. ,  an  opportunity  to  practice  the  skills  and  receive  real-time 
feedback). 


7.  I  was  satisfied  with  the  quality  and  organization  of  these  modules?  7 


8.  I  was  satisfied  with  the  amount  that  I  learned  during  the  training.  7 


Interactions  with  your  Consultant  and  Scheduling 
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9.  During  role-plays  and  throughout  the  training  program,  you  had  the 
opportunity  to  ask  your  consultant  questions  about  the  content  of  the 
curriculum  or  rationale  for  certain  skills  during  the  role  plays.  1  was  satisfied 
with  my  consultant’s  ability  to  answer  my  questions. 

7 

10. 1  am  satisfied  with  the  interactions  that  1  had  with  my  consultant  (e.g.,  time  to 
respond  to  questions,  weekly  updates). 

7 

1 1 .  The  training  program  was  arranged  so  that  you  could  proceed  at  your  own 
pace.  1  was  satisfied  with  option  of  completing  the  program  at  my  own  pace 

7 

12. 1  was  satisfied  with  the  flexibility  of  scheduling  meetings  and  role  plays. 

7 

Overall  Training  Program 

13.  During  role-plays  and  throughout  the  training  program,  you  had  the 
opportunity  to  ask  your  consultant  questions  about  the  content  of  the 
curriculum  or  rationale  for  certain  skills  during  the  role-plays.  1  was  satisfied 
with  the  consultant’s  ability  to  answer  my  questions. 

7 

14.  Overall,  1  was  satisfied  with  this  program. 

7 

15.  Would  you  recommend  this  type  of  web-based  instruction  to  other  individuals 
who  are  not  able  to  receive  on-site  instruction? 

Yes 

4.  Recruit,  pretest,  and  train  and  treat  5 
parents  and  5  children  in  the  technology- 
enhanced  test  group  and  5  parents  and 
5  children  in  the  waitlist-control  group 
across  two  cohorts.  Schematic  2  details 
the  progress  for  the  technology- 
enhanced  parent  training  and  in-home 
early  intervention  for  children  with  autism 
(Experiments  1  &  3). 

Schematic  2.  - ± 


Parent  Training 

-  Randomized  Clinical  Trial 

- 0  Completed 

- 0 

- 0 

-  Enrolled 

- Test  =  1 

- Control  =  1 

-  Completing  Pre-Tests 


- :  Single-Subject  Design 

I - 1 

Child  Early  Intervention  Services 

-  Randomized  Clinical  Trial 

-  Completed 

- 0 

- 0 

-  Enrolled 

- Test  =  1 

—  Control  =  1 
-  Completing  Pre-Tests 


- Single-Subject  Design 


0 
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a.  Randomized  Clinical  Trial.  We  currently  have  one  parent  and  one  child 

randomized  to  the  test  and  waitlist-control  groups.  However,  due  to  recent  efforts 
to  disseminate  information  about  our  program,  we  have  30  parents  and  15 
children  at  various  stages  of  completing  the  pre-tests  and  pre-assessments 
required  prior  to  randomization.  This  notable  increase  in  participants  is  likely  due, 
in  part,  to  the  recruitment  effort  summarized  below. 

i.  Communication  with  EFMP  coordinators.  We  have  created  a  visual  map  of 
all  the  EFMP  coordinators  in  the  United  States  in  Figure  6.  For  each  pin  on 
the  map,  we  have  identified  the  contact  information  of  the  EFMP  coordinator 
and  the  associated  military  base.  The  red  pins  denote  bases  that  have 
returned  our  phone  calls  or  emails  and  have  offered  to  distribute  our  materials 
to  military  families  that  may  benefit  from  our  services.  The  blue  pins  denote 
bases  that  have  not  returned  our  communication  attempts.  As  indicated  by 
the  number  of  red  pins,  we  have  successfully  contacted  and  distributed  our 
recruitment  materials  to  over  77%  of  the  bases. 

Figure  6. 


ii.  We  delivered  a  webinar  on  the  grant  to  26  EFMP  coordinators  and  medical 
personnel  in  the  Navy,  which  was  arranged  by  EFMP-Analyst-Commander 
Lisa  Davis;  we  also  sent  our  flyers  and  information  papers  to  10  Regional 
Case  Liaisons.  All  attendees  expressed  enthusiasm  for  the  program  and  its 
potential  for  improving  the  lives  of  military  families  that  they  serve. 


14 


iii.  We  conducted  a  webinar  for  medical  personnel  and  EFMP  coordinators  at 
Tinker  Air  Force  Base. 

iv.  Beverly  Boaz,  SME  Senior  Consultant  in  BUMED,  worked  with  us  to  publish 
information  about  the  grant  in  Health  Care  Operations  Newsletter,  which  is 
distributed  monthly  to  hundreds  of  military  families. 

v.  We  spoke  with  Elizabeth  Leath,  who  is  a  Family  Support  Specialist  at  the 
Airman  &  Family  Readiness  Center  in  the  United  Kingdom.  In  addition,  we 
conducted  a  webinar  with  the  EFMP  coordinator  and  medical  personal  at  the 
military  base  associated  with  RAF  Lakenheath  in  the  United  Kingdom. 
Although  we  would  be  limited  in  providing  in-home  early  intervention  services 
abroad,  there  is  strong  demand  for  parent-training  services. 

vi.  We  have  spoken  with  the  Association  for  University  Centers  on  Disabilities 
(AUCD)  to  disseminate  information  about  the  grant.  We  have  spoken  with 
several  medical  personnel  who  are  responsible  for  diagnosing  children  with 
autism;  as  a  result,  this  may  lead  to  families  learning  about  the  grant  and 
contacting  us.  We  are  working  with  Leadership  Education  in 
Neurodevelopment  and  Related  Disabilities  (LEND)  programs  to  share 
information  about  the  grant. 

vii.  As  another  means  for  military  families  to  learn  about  the  grant,  we  have 
asked  military-housing  communities  near  bases  to  distribute  our  recruitment 
materials.  Of  the  149  bases  that  we  have  contacted,  49  responded  to  our 
requests  and  have  distributed  our  materials. 

viii.  We  have  contacted  244  military  veteran  associations  or  centers  throughout 
the  United  States  to  distribute  our  recruitment  materials  because  they  may 
know  of  military  families  with  a  child  with  autism.  130  centers  have  agreed  to 
distribute  and  post  information  about  our  grant. 

ix.  We  have  contacted  the  Department  of  Defense  Education  Activity  (DoDEA) 
because  we  hope  to  disseminate  information  to  military  schools  who  may 
serve  children  with  autism.  In  addition,  we  have  contacted  preschool  and 
early-intervention  programs  near  military  bases  in  the  case  that  the  teachers 
may  know  of  military  families  who  could  benefit  from  participating  in  the  grant. 
We  are  also  working  with  Centers  for  Excellence  in  Disabilities  in  different 
states  to  spread  information  about  the  grant. 


15 


Key  Research  Accomplishments 

We  have  completed  evaluations  for  the  technology-enhanced  parent-training 
(Experiment  1)  and  tutor-training  (Experiment  2)  curricula  using  single-subject 
experimental  designs.  All  results  supported  the  efficacy  of  the  web-based  teaching 
procedures.  The  initial  results  from  the  randomized  clinical  trial  for  training  tutors  also 
supported  the  efficacy  of  the  web-based  teaching  procedures  for  improving 
performance  on  both  primary  dependent  measures:  The  differences  between  the  test 
and  control  groups  approached  statistical  significance  for  the  BISWA  (p  =  .066)  and  the 
differences  for  the  BISPA  were  highly  significant  (p  <  .0001 ).  Given  our  recent 
recruitment  efforts,  we  plan  to  initiate  the  randomized  clinical  trials  for  technology- 
enhanced  parent-training  (15  assigned  to  the  test  group;  15  assigned  to  the  control 
group)  and  in-home  early  intervention  services  (7  assigned  to  the  test  group;  7  assigned 
to  the  control  group). 
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Reportable  Outcomes 

We  have  presented  the  preliminary  results  shown  in  Figure  3  above  (training  ABA  tutors 
to  conduct  a  preference  assessment  using  web-based  training)  at  a  national 
conference,  the  Annual  Conference  of  the  Association  for  Behavior  Analysis 
International.  We  are  currently  writing  a  paper  describing  this  study  to  be  submitted  to 
the  Journal  of  Applied  Behavior  Analysis  for  consideration  for  publication. 
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Conclusions 

Although  the  results  of  our  preliminary  studies  and  the  early  results  of  the  randomized 
clinical  trial  are  encouraging,  it  is  premature  to  draw  any  firm  conclusions  at  this  juncture 
of  the  award. 
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